The technical field concerns image processing. More particularly, the technical field covers automated analysis of magnified images. More particularly still, the technical field concerns an instrumentation system in which magnified images of material are subjected to image processing in an image processing system with a graphical user interface that enables a user to selectively select and initialize an image analysis algorithm and to screen results.
Magnified images of biological material are obtained for purposes of study, diagnosis, or determination of experimental results. Such images may be obtained by instrument systems from material disposed in multi-well plates, plastic or glass culture dishes, cells disposed on plastic or glass slides, and/or tissue sections mounted to plastic or glass slides. The magnified images are subjected to analysis by means of an automated image processing system constructed to execute image processing algorithms in order to determine characteristics of the material important to the intended purposes.
An automated image processing system incorporating inventions described in U.S. patent application Ser. No. 12/454,081 and U.S. patent application Ser. No. 12/459,146 has contributed a number of key advances to image cytometry, high-content screening, and high-content analysis software packages. The system is designed for ease of use, accessibility and scalability and includes a graphical user interface (“GUI”) for analysis and data viewing that is much more accessible for the non-expert user than previous tools. The image input and numerical data structures of the system conform to standard open formats and the system works natively on commercially-available versions of standard operating systems. Multithreading support enables speed and straightforward scale up. This system is designed to automate the analysis of images from digital microscope cameras and high-content screening analysis instruments. For most assays, an assay development step is required to determine the best image analysis settings and biological preparation. It is an iterative plate-by-plate and well-by-well process cycling between image acquisition, image analysis and statistics. Once the assay conditions and image processing conditions are set, these settings are applied in more routine conditions.
In the automated image analysis of the pending applications, analysis starts by breaking down each image into core biological component masks for cells and tissues. Then measurements are aggregated as needed for experiments with slides, wells, plates etc. First, all of the nuclei available from the nuclear images are identified. A nuclear mask for each cell is established where the mask contains all of the pixels locations automatically identified as nuclear for a given cell. Then a second image is analyzed. Presuming an RNA image for example, analysis assigns RNA spots in the image to an RNA mask. These masks are determined by the algorithm but roughly correspond to the brightest pixels in the RNA image. A rich set of data parameters are then calculated on a “per cell basis”.
As per FIG. 1, an RNA example of the cell is analyzed, wherein Nm is a nuclear mask and corresponds to the number of pixels that make up the nuclei, Cm is a cytoplasmic mask, which extends from the cell boundaries to the nucleus, and Rm is an RNA mask and corresponds to the number of pixels found within RNA dots.
In an automated image analysis system incorporating inventions of the pending '081 and '146 applications, system functionality is presented to users through a GUI including a main window, an image window, and a data viewer.
The main window, seen in FIG. 2, is what users see at launch. The image viewer and data are accessed from this main window. As may be appreciated, the main window constitutes a simple, easily understood and operated interface. This simplicity is especially stark relative to other similar packages. The steps required to analyze a slide, a single well, a multi-well plate, or an arbitrarily large batch of multi-well plates are similar. In the simplest case, a user selects an appropriate image processing algorithm, indicates a storage location of an image or images to be processed, and clicks a ‘run’ button.
The ‘Thread’ count displayed in this window refers to how many of the computer's processors are to be used for analysis. This multithreading capability is useful in providing speed and scalability for larger data sets and more sophisticated image analysis. Multithreading is useful for common multi-core laptops and workstations as well as for larger scale dedicated servers with many processors.
Once the initial analysis is complete, image processing results may be inspected in an image viewer. See FIG. 3 in this regard. This window displays raw unprocessed images overlaid with masks created by automated image processing. an original image. Here again, the emphasis is on usability and simplicity. Every cell is identified with a unique number and the results of the image segmentation are clearly displayed. The controls offer instant updates as to which image, well, mask or mask edge is displayed and transparently integrate well with image zooming, contrast enhancement, screenshot, pseudo-color control features. The result is a facile interface enabling a user to verify image segmentation and inspect areas of interest within a screen. The example shown in FIG. 3 is a two-color RNA example with automatically generated cell masks, unique cell IDs and control dialog windows.
The numbers derived from the segmentation are then viewed and manipulated through a data viewer. See FIG. 4. Though hundreds of measurements may be routinely available during image processing, a far smaller number may be relevant to any one experiment. To afford ease of use, the data viewer is built to offer access to measurements of interest without overwhelming users. Users can create and export raw data tables, histograms, scatter plots. All of the data can be filtered through a very powerful gating interface.
Per FIG. 4, automated image analysis generates a number of measurements for each cell within an image. These data are aggregated and analyzed as required by the experiment within the data viewer.
The data viewer of FIG. 4 enables inspection of numerical data for each individual cell, well, plate and experiment. For example, the automated image processing systems of the referenced provisional applications may work with plates up to 3456 in size, and may store an experimental condition for each well. Such conditions may include, for example, time points in a time course or chemistry dose information. All the data are stored and can be exported into commonly used text formats (csv, jpeg files).
While the automated image analysis system incorporating inventions of the pending '081 and '146 applications provides fast, intuitive user control and management of automatic image processing, data calculation, and result presentation in high content screening and/or high throughput screening, further functionality and adaptability are desirable in order to enrich the results produced.
It is desirable for a gating interface to be able to take advantage of automatic segmentation of the raw images into an aggregation of individual cell data. Once the data is transformed into cytometry data, the ability to filter the cells into subpopulations provides a useful analysis tool.
It is desirable to be able to selectively initialize image processing by selecting or assembling an image processing algorithm from a library of routines.
It is desirable to be able to selectively establish parameters for execution of the algorithm.
It is desirable to be able to screen images by visual inspection before, during, and after image and data processing by the algorithm in order to assess the quality of results obtained.